Papers with aggregation techniques
Dependency Tree Annotation with Mechanical Turk (D19-59)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourcing is often used to obtain linguistic annotations but is rarely used for parsing. |
| Approach: | They propose to use Mechanical Turk to crowdsource parse trees using an interactive graphical dependency tree editor. |
| Outcome: | The proposed method is the first published use of Mechanical Turk to crowdsource parse trees . the authors find that the workers achieve high levels of accuracy on 72% of the sentences . |
An Individualized News Affective Response Dataset (2024.acl-srw)
Copied to clipboard
| Challenge: | a new dataset captures subjective affective responses to news headlines . current methods to assess emotion detection ignore subjective differences in groups and individuals . |
| Approach: | They propose a large-scale dataset capturing subjective affective responses to news headlines . the dataset includes Facebook post screenshots from popular UK media outlets . |
| Outcome: | The proposed dataset captures subjective affective responses to headlines from popular media outlets. |
SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Large language models have a tendency to make confidently wrong predictions, highlighting the need for uncertainty quantification (UQ) . previous studies focused on aleatoric uncertainty, but the full spectrum of uncertainties, including epistemic, remains inadequately explored. |
| Approach: | They propose a method to quantify uncertainty in large language models (LLMs) they use a set of perturbations and an aggregation module to generalize the method. |
| Outcome: | The proposed method improves model uncertainty calibration and reduces expected calibration error by 50% on average. |
Rhombus: Incentivizing Coordination in Parallel Thinking through Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Ziyuan Nan, Qi Yi, Di Huang, Yutong Wu, Guanhua Huang, Xue Gong, Kejiao Li, Yuhao Jiang, Chenchen Zhang, Zenan Xu, Xing Hu, Bo Zhou
| Challenge: | Parallel thinking is a promising avenue for scaling test-time compute in Large Language Models . however, coordinating the exploration and aggregation stages remains challenging . |
| Approach: | They propose a parallel thinking framework that explicitly incentivizes coordination between components via end-to-end reinforcement learning. |
| Outcome: | The proposed framework improves accuracy by 6.0% over long chain-of-thought baselines while reducing wall-clock latency by 39.4% under matched token budgets. |